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The  amount  of  work  we  have  done  and  the  number  of  subjects  studied  in  one  year  was 
far  beyond  what  was  promised  in  the  original  two  year  proposal  [1].  The  original  proposal 
focussed  on  the  development  and  analysis  of  single-sensor  parameter  estimation  schemes 
for  ARMA  signals  in  noise.  In  the  single  sensor  problem,  we  solved  several  new  problems 
for  these  signals  and  also  added  sine  wave  signals  in  noise.  Furtheremore,  we  extended 
the  research  to  multi-sensor  (or  sensor  array)  estimation  algorithms  which  are  useful  for 
direction-of-arrivals  estimation.  All  of  the  proposed  algorithms  have  been  tested  by  computer 
simulation  to  verify  their  operation. 

A  testimony  to  the  significance  and  recognition  of  our  recent  research  is  that  we  were 
invited  to  publish  the  article  [9j  as  a  chpater  in  a  forthcoming  book  edited  by  S.  Haykin  [101. 

The  research  results  obtained  this  year  are  described  in  detail  in  the  references  [2l-[91. 
The  work  in  all  of  these  articles  was  performed  with  the  support  (either  whole  or  partial) 
of  the  AFOSR  grant,  and  this  financial  support  is  acknowledged  in  the  articles  themselves. 
The  work  is  summarized  in  the  following. 
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Annual  Report 

Research  Under  Grant  AFOSR-88-0080 
November  1,  1987  to  November  1,  1988 

1  Introduction 

This  report  describes  the  work  supported  by  the  Air  Force  Office  of  Sceintific  Research  under 
Grant  AFOSR-88-0080  during  the  first  year  of  a  two  year  period.  The  grant  supported  the 
work  of  the  Principal  Investigator  Professor  Arye  Nehorai  and  of  his  student  Mr.  David 
Starer  at  Yale  University. 

The  amount  of  work  we  have  done  and  the  number  of  subjects  studied  in  one  year  was 
far  beyond  what  was  promised  in  the  original  two  year  proposal  [l].  The  original  proposal 
focussed  on  the  development  and  analysis  of  single-sensor  parameter  estimation  schemes 
for  ARMA. signals  in  noise.  In  the  single  sensor  problem,  we  solved  several  new  problems 
for  these  signals  and  also  added  sine  wave  signals  in  noise.  Furtheremore,  we  extended 
the  research  to  multi-sensor  (or  sensor  array)  estimation  algorithms  which  are  useful  for 
direction-of-arrivals  estimation.  All  of  the  proposed  algorithms  have  been  tested  by  computer 
simulation  to  verify  their  operation. 

A  testimony  to  the  significance  and  recognition  of  our  recent  research  is  that  we  were 
invited  to  publish  the  article  [9]  as  a  chpater  in  a  forthcoming  book  edited  by  S.  Haykin  [10]. 

The  research  results  obtained  this  year  are  described  in  detail  in  the  references  [2]- [9]. 
The  work  in  all  of  these  articles  was  performed  with  the  support  (either  whole  or  partial) 
of  the  AFOSR  grant,  and  this  financial  support  is  acknowledged  in  the  articles  themselves. 
The  work  is  summarized  in  the  following. 


2  Single  Sensor  Algorithms 

2.1  Algorithm  Development 

A  convenient  model  for  time-series  analysis  and  system  identification  is  the  autoregressive 
moving-average  (ARMA)  or  rational  transfer  function  model:  In  time-series  analysis,  a  mea¬ 
sured  signal  can  be  modeled  as  the  output  of  a  rational  transfer  function  driven  by  white 
noise.  The  estimated  modH  can  then  be  used  to  determine  the  signal  spectrum.  In  «yst<«tn 
identification,  an  unknown  system  can  be  modeled  as  a  rational  transfer  function  whose  pa¬ 
rameters  can  be  estimated  from  measurements  of  the  input  and  output  signals  using  ARMA 
identification  algorithms.  Knowledge  of  these  parameters  can  be  used,  for  example,  by  an 
adaptive  regulator  to  control  the  system. 

Most  existing  ARMA  estimators  employ  “unconstrained”  rational  models.  However,  in 
many  applications  the  measured  signal  is  known  to  have  special  properties,  and  it  is  prefer¬ 
able  to  constrain  the  model  conform  to  these  properties.  Furtheremore,  most  existing 
ARMA  estimators  ignore  the  contamination  of  the  signal  by  noise,  and  this  leads  to  incor¬ 
rect  estimates.  To  solve  these  problems,  we  developed  in  [2j  a  general  family  of  ARMA 
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adaptive  algorithms  which  utilize  a  prion  known  information  concerning  the  signal's  prop¬ 
erties  and  solve  the  problems  associated  with  non-linearities  which  arise  from  the  presence 
of  noise.  These  algorithms  can  track  time-varying  parameters,  are  more  accurate  arid  are 
computationally  more  efficient  than  the  unconstrained  ARMA  algorithms.  Applications  of 
these  algorithms  include  signal  analysis  in  multipath  scenarios,  image  deblurring,  structured 
signal  (e.g.  band-pass)  spectrum  analysis. 

In  many  ARMA  spectral  estimation  problems,  the  parameters  of  interest  are  the  poles 
(or  singularities)  of  the  rational  transfer  function.  These  are  useful,  for  example,  for  spectral 
peak  estimation,  robust  speech  communication,  and  directions  of  arrivals  (DOA's)  estima¬ 
tion.  Thus  there  exists  a  need  for  an  algorithm  which  can  satisfy  the  simultaneous  require¬ 
ments  of  a)  directly  providing  estimates  of  a  system’s  poles  and  of  b)  providing  new  pole 
estimates  as  each  data  sample  is  received.  Our  research  in  [3]  has  provided  an  algorithm 
which  satisfies  the  above  requirements.  In  this  method,  the  system  is  parametrized  directly 
in  terms  of  its  poles.  Convergence  analysis  is  provided  and  it  is  shown  that  the  algorithm 
converges  to  the  true  parameters.  The  research  has  already  provided  some  spinoff  results  in 
the  form  of  new  polynomial  factorization  algorithms  which  will  be  published  at  a  forthcoming 
conference  [4], 

One  of  the  common  problems  in  system  identification  is  that  the  noise  characteristics  may 
not  be  known  or  may  vary  from  application  to  application.  In  [5]  we  have  develped  a  new 
system  identification  algorithm  that  is  robust  against  unknown  noise  properties.  The  method 
is  an  optimal  min-max  instrumental  variable  (IV)  method  that  gives  the  smallest  estimation 
error  variance  in  the  worst  noise  case  from  a  prespecified  class  of  noises.  Implementation  ol 
the  new  method  does  not  require  knowledge  of  system  or  noise  parameters  and  therefore  can 
be  done  without  iteration.  It  is  useful  for  example  for  dead-beat  controllers  where  there  is 
no  need  for  the  noise  parameters  to  be  estimated.  This  method  was  applied  successfully  to 
both  artificial  and  real  data  from  gas  furnace  mesurements. 

2.2  Performance  Analysis  of  Algorithms 

Performance  analysis  includes  derivation  of  the  error  covariance  matrices  of  estimation  al¬ 
gorithms  and  derivation  of  generic  bounds  on  estimation  accuracy.  A  common  bound  is  t  lie 
Cramer-Rao  bound  (CRB)  which  is  a  lower  bound  on  the  estimation  error  covariance  marix 
of  any  unbiased  estimator.  The  CRB  is  useful  for  evaluating  the  quality  of  algorithms  and 
for  gaining  insight  into  the  problem  of  interest. 

The  main  difficulty  in  deriving  the  covariance  marix  of  estimation  algorithms  stems  from 
their  estimation  non-linearities.  In  [6]  we  have  provided  a  statistical  analysis  of  t  wo  non¬ 
linear  least-squares  estimators  (NLSF-’s)  of  sine  wave  parameters  in  the  colored  noise  case. 
These  estimators  are  the  basic  NLSE,  which  ignores  the  possible  correlation  of  the  noise, 
and  the  optimal  NLSE.  which  estimates  the  noise  correlation  (appropriately  paramet  fixed ) 
as  well  as  the  sine-wave  parameters.  It,  is  shown  that  these  two  NLS  estimators  have  the 
same  accuracy  in  large  samples.  This  result  provides  complete  justification  for  preferring 
the  computationally  less  expensive  basic  NLSE  over  the  “optimal'  NLSE.  Both  estimators 
are  shown  to  achieve  the  Cramer-Rao  bound  as  the  sample  size  increases.  A  simple  explicit 
expression  for  the  asymptotic  CRB  matrix  is  provided,  which  should  be  useful  in  studying 
the  performance  of  sine-wave  parameter  estimators  designed  to  work  in  the  colored  noise 
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case. 

One  of  the  well  known  methods  for  sine  wave  frequency  estimation  is  the  Pisarenko 
method  (see  corresponding  reference  in  [7]).  The  method  consists  of  determining  the  min¬ 
imum  eigenvalue  of  the  data  covariance  matrix  and  its  assoicatied  eigenfilter  zeros.  In  [7] 
we  provide  a  self-contained  statistical  analysis  of  this  method.  An  explicit  formula  is  pro¬ 
vided  for  the  asymptotic  covariance  matrix  of  the  method.  This  covariance  is  then  compared 
with  the  Cramer-Rao  bound  and  the  Yule- Walker  method  (see  [7]).  Our  results  show  that 
both  the  Pisarenko  and  the  Yule- Walker  methods  are  quite  inefficient  in  the  statistical  sense. 
Their  error  variances  are  much  larger  than  the  CRB.  It  is  also  shown  that  the  variances  of 
these  two  methods  increase  faster  than  the  CRB  when  the  signal-to-noise  ratio  decreases. 
Since  both  the  Pisarenko  and  the  Yule-Walker  methods  are  quite  inefficient  statistically,  they 
are  attractive  only  when  computational  simplicity  is  a  must. 

3  Sensor  Array  Processing 

Sensor  array  processing  is  a  common  name  for  several  problems  associated  with  array  of 
sensors.  The  sensors  may  be  electromagnetic  antennas,  microphones,  geophones,  underwater 
sonars,  etc.  These  sensors  are  arranged  in  an  arbitrary  geometry  to  form  a  sensor  array.  The 
sensor  array  records  signal  waveforms  of  distant  emitters.  These  in  turn  are  used  to  estimate 
unknown  emitter  parameters,  such  as  directions  of  arrivals,  location,  signal  waveforms,  etc. 
The  great  interest  in  sensor  array  processing  stems  from  their  wide  applicability  in  radar, 
sonar,  radio  and  microwave  communication,  seismology  and  hydroacoustics.  An  interesting 
note  is  that  damped/undamped  sinewave  parameter  estimation  in  noise  turns  out  to  be  a 
special  case  of  the  sensor  array  processing  problem. 

The  sensor  output  data  is  modeled  as  a  superposition  of  individual  wave  forms.  In 
the  narrow-band  case  (i.e.  when  the  power  of  all  the  emitter  signals  is  in  the  same  narrow 
frequency  band)  the  sensor  data  model  is: 

y(t)  =  A{0)x(t)  +  e(t)  t  =  1,2, . . . ,  iV  (1) 

where  (y(t)}  are  the  observed  sensor  data  vectors,  {x(t)}  are  the  unknown  signal  wave  form 
vectors,  and  e(<)  is  an  additive  noise.  The  matrix  A(0)  models  the  characteristics  of  the 
array  (e.g.  geometry  and  sensor  directivity).  Based  on  the  sensor  observations  |i/(f)}.  the 
parameters  of  interest  (9  and  (i(t)})  are  estimated. 

Two  classes  of  methods  for  estimating  the  unknown  parameters  of  (If  have  received 
significant  attention  in  recent  years.  The  first  is  the  multiple  signal  characterization  (MUSIC) 
method  which  is  based  on  the  eigendecompostion  of  the  data  sample  covariance  matrix  and 
is  in  fact  an  ad-hoc  method.  The  second  is  the  maximum  likelihood  (ML)  which  is  based  on 
statistical  foundations. 

In  the  following  we  summarize  our  research  results  in  the  area  of  sensor  array  processing 
on  these  algorithms  and  the  corresponding  CRB. 

3.1  Algorithm  Development 

The  problem  of  estimating  the  direction  of  arrivals  of  far-field  source  wave  forms  by  uniform 
linear  arrays  and  the  related  one  of  estimating  the  parameters  of  multiple  exponential  signals 
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in  noise  is  important  in  many  applications  of  signal  processing.  As  yet,  no  optimal  estima¬ 
tion  methods  tailored  to  this  problem  with  guaranteed  convergence  have  been  found  since 
the  estimation  involves  the  difficult  problem  of  maximizing  the  highly  nonlinear  likelihood 
function.  Our  research  in  [8]  presents  an  ML  solution  to  the  problem.  Using  the  properties 
of  shift  matrices  we  derive  simple,  exact,  closed-form  expressions  for  the  gradient  vector 
and  the  Hessian  matrix.  Based  on  these  results,  a  Newton  algorithm  is  presented  tor  the 
estimation  of  source  DOA’s  and  multiple  exponential  signals  in  noise.  The  main  advantages 
of  this  algorithm  include  ease  of  implementation  with  matrix  software  packages  (such  as 
MATLAB).  as  well  as  fast  and  guaranteed  convergence  to  a  local  maximum  of  the  likelihood 
function.  A  globally  convergent  ML  algorithm  is  currently  being  deveolped. 

3.2  Perfromance  Analysis  of  Algorithms 

Since  the  MUSIC  is  computationally  simpler  than  the  ML  method,  and  since  these  methods 
are  widely  used,  it  has  been  of  great  interest  and  importance  to  evaluate  the  performance  of 
these  two  methods,  comparing  them  with  one  another  and  with  the  CRB. 

Our  paper  [9]  provides  extensive  studies  of  the  performance  of  the  MUSIC  and  ML 
methods,  and  analyzes  their  statistical  efficiency.  Closed-form  expressions  are  derived  for 
the  MUSIC  covariance  matrix  and  the  CRB  for  the  estimation  problem  (1).  It  is  shown  that 
unless  the  emitter  signals  are  uncorrelated  (i.e.  absence  of  multipaths)  and  the  number  of 
sensors  is  large,  the  MUSIC  estimator  cannot  achieve  the  CRB.  The  relationship  between  the 
MUSIC  and  the  ML  is  also  investigated.  It  is  shown  that  the  MUSIC  is  a  large  sample  (i.e. 
large  N)  realization  of  the  ML  estimate  if  and  only  if  the  emitter  signals  are  uncorrelated. 
The  paper  also  contains  results  on  the  resolvabilitv  of  the  MUSIC  and  ML  algorithms  for 
the  problem  of  finding  the  DOA’s  of  plane  waves  using  a  uniform  linear  array. 

One  of  the  well  known  results  in  estimation  theory  is  that  the  ML  method  achieves  the 
CRB  asymptotically.  Our  results  prove  that,  for  the  problem  (1),  the  ML  method  cannot 
achieve  the  CRB  unless  the  number  of  sensors  is  very  large,  even  if  the  number  of  data  (A’)  is 
large.  This  '‘shocking”  result  is  explained  in  [9].  It  raises  the  question:  Do  better  estimators 
than  the  ML  method  exist  for  a  finite  number  of  sensors?  This  question  is  currently  boinu' 
investigated. 


4  Future  Outlook 

Our  research  on  algorithm  development  and  analysis  supported  by  this  grant  is  rapidly 
growing.  In  the  first  year  of  this  grant  we  have  solved  in  [2]-[9]  many  important  and  difficult 
problems  which  currently  exist  in  the  signal  processing  and  system  identification  areas.  Most 
of  these  problems  were  not  raised  in  the  original  proposal.  However,  t  here  are  many  problems 
that  we  still  have  to  address. 

In  the  future  we  will  continue  the  development  and  analysis  of  algorithms  for  signal 
processing  and  system  identification.  We  will  derive  the  ML  covariance  matrix  for  (1)  and 
analyse  methods  other  than  MUSIC  and  the  MLE,  such  as  weighted  MUSIC,  ESPRI  T  and 
subspace  rotation  methods  (SRM’s).  We  will  consider  sensor  array  models  that  are  more 
general  than  (1).  For  instance,  models  of  near-field  sources,  correlated  sensor  noises,  signal- 
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modeled  through  their  covariances  rather  than  their  amplitudes,  and  finite  data  models.  We 
will  develop  new  algorithms  and  analyse  their  performance  for  these  problems.  A  globally 
convergent  ML  algorithm  for  (1)  will  be  developed.  Optimal  methods  that  could  outperform 
the  ML  method  will  be  developed  for  a  finite  number  of  sensors  and  data  samples.  Bounds 
that  are  more  appropriate  than  the  CRB  will  be  considered  for  the  problems  of  closely  spaced 
emitters  and  damped  (transient)  signals. 

The  support  of  the  Air  Force  Office  of  Scientific  Research  has  been  essential  for  the 
research  in  [2]-[9]  and  is  gratefully  acknowledged.  We  hope  that  this  support  will  continue 
beyond  the  next  year  and  will  include  more  students  to  solve  more  problems. 
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